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ABSTRACT 



This thesis describes the Testing system, a CAI system which monitors the 
progress of a student and automatically adjusts the lesson to suit his/her understanding 
of the material being presented. Each lesson is divided into three segments. The first 
segment consists of questions critical to the basic understanding of the lesson, the 
second questions complementary to the lesson, and the third questions supplementary. 
After the first segment, the student's performance is evaluated and a decision is made 
whether or not he/she should continue on to the complementary segment of the lesson 
or go on to the next lesson. Similarly, in the second segment, a decision whether to 
continue on to the supplementary segment or to the next lesson is made. Thus, a 
proficient student can progress through the material quickly, a slow learner slowly. 
The questions are grouped into types, and each of them is ranked as an easy or 
difficult concept to provide more precise guidance for the student's improvement. An 
example lesson is provided as an illustration. 
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I. INTRODUCTION 



Over the years, the computer has become an indispensable tool in all walks of 
life. One of its important applications has been in automated instruction. It has led to 
a field of research called Computer-Aided Instruction (CAI). CAI is a powerful 
methodology for enhancing the overall quahty and effectiveness of education. Many 
CAI systems have been developed for a variety of knowledge domains. Experience has 
shown that such systems often cause annoyance to fast learners and, at the other 
extreme, discouragement to the slow ones. Moreover, since learning is dynamic and 
non-uniform, it is desirable for an instructional and diagnostic system to adjust itself to 
the student. This helps accommodate the student's familiarity and talents for different 
topics so that an intelligent/knowledgeable student is not surfeited with redundant 
information, nor a student who has extreme difficulties in a topic be dampened of 
motivation. 

This thesis proposes a CAI system which includes a diagnostic model to test how 
well the learner has absorbed the instructional material and diagnose precisely where 
the learner has difficulties. The diagnostic model being implemented here was first 
developed by Professor Taracad R. Sivasankaran and Professor Tung Bui [Ref. 1] and 
based on the Bayesian Analysis. 

The proposed system for Computer-Aided Instruction separates the questions 
into three segments. The first segment consists of questions which are critical to the 
understanding of the lesson. The second and the third segments consists of questions 
which are complementary and supplementary to the critical segment. By separating the 
questions in this manner, the system can determine after the learner has completed the 
critical questions whether or not the student has mastered the material. The system 
determines the student's knowledge by examining the number of correctly answered 
questions and uses this information to calculate the (posterior) probability of the 
student's ability to answer the complementary and supplementary questions. If this 
probability exceeds the upper cut-off criterion, the student is assumed to have mastered 
the lesson and may continue on to the next lesson. However, if the probability is below 
the lower cut-off criterion, the student is deemed to have insufficient knowledge about 
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the lesson and the system will not allow the student to continue with lesson. If the 
probability is in between the two cut-off criteria, the student's knowledge of the lesson 
cannot be determined by the student's performance; thus, further testing/questioning is 
required. The system will then present the student with a question from the 
complementary segment. After the student completed the test question, the same 
probabilistic analysis is applied and a decision is made as to whether or not the student 
should answer the supplementary questions or go on to the next lesson. 

Chapter 2 provides the necessary background concerning the general description 
of Computer-Aided Instruction, the basics of the Bayesian Analysis, and demonstrates 
the design of the questions for the Testing System. For illustration purposes, the 
questions are drawn from the "Student Pilot's Flight Manual" [Ref 2]. Chapter 3 
presents the diagnostic model. Chapter 4 describes the system architecture, data 
structures, the detailed design and descriptions of the program. Finally, Chapter 5 
concludes the thesis and suggests future directions. 
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II. BACKGROUND 



A. COMPUTER-AIDED INSTRUCTION 
1. Deflnition 

The tenn CAI (Computer-Aided Instruction) is commonly used to refer to aU 
aspects of computers in education. The definition of CAI has been narrowed and 
considered, a distinct subject within a broader category termed Computer Based 
Education (CBE). While the actual boundaries of the defmition of CAI are still 
somewhat blurred, a general consensus of what constitutes a CAI has evolved: 

"CAI is a term used to described the process of interactive teaching with the 
aid of a computer. The process directly involves the computer in the storage and 
presentation of instructional materials to the student. Both written material and 
graphics are presented to the student in a logical manner". [Ref 3] 

This definition, while broad, serves to distinguish CAI from another growing 
field of computers in education termed Computer Managed Instruction (CMI). In 
CM I, the computer is not normally used as a direct interface with the student to effect 
learning. Instead, the computer is used as a management tool to assist the instructor. 
The materials presented to the student in the CMI model are not of a direct 
educational nature and include such things as tests and management of the flow of a 
student's education. The computer is used as a gate to control student progress and to 
provide extensive managerial data to the instructor to enable more individual tailoring 
of instruction. 

Despite the fact that a few current authors continue to refer to both CAI and 
CMI as a single entity. There are two important distinctions between CAI and CMI: 

• CAI is usually performed in an on-line real time or near real time basis in order 
to provide immediate feedback of an instructional nature to the student. CMI, 
on the other hand, is typically not a real time process and focuses on the batch 
processing of data for later presentations to the instructor. [Ref 4] 

• CAI is more intensive both in hardware and software [Ref 5]. In order to 
implement a successful CAI system, students must have ready access to 
computer terminal, necessitating a significant investment in hardware. In 
addition, the software (which the company refers to as courseware) required to 
effect a CAI is complex and long lead times and high development costs are the 
rule. In contrast, CMI can be effective with only a few terminals for joint 
instructor and student use. Software development for CMI tends to be 
relatively straight forward and relatively inexpensive. 
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2. Basic Categories of CAI 

CAI can be further subdivided into five teaching strategies: (1) Drill and 
practice; (2) Tutorial; (3) Problem solving tasks; (4) Simulation; and (5) Instructional 
games [Ref, 6], 

a. Drill-and- Practice 

The user is assumed to have already learned the basis of the material to be 
presented through traditional classroom lecture techniques. The goal of computer is to 
cement the retention of the material by repeated practice and exercise. This method 
provides the student with the capability to work through the large array of problems 
and to develop an intuitive understanding of the procedures used. [Ref 7] 

Drill-and-Practice programs tend to be organized linearly with a brief 
introduction followed by the presentation of a series of problems or solution step. A 
series of preselected problems can be provided from outside material by the student 
through appropriate interfacing with the program. The latter method is more flexible 
and provides for more student involvement. The embedding of the actual problems in 
the software significantly restricts the scope of the program and requires extensive 
maintainance to change the problem sets. DriU-and -Practice type software is 
particularly well suited to courses where a repetitive practice is required to learn a skill 
or a concept. The linear sequencing of solution steps for many mathematics and 
statistical problems can easily be accommodated using the Drill-and-Practice strategy. 

b. Tutorial 

The tutorial strategy encompasses a great deal more than the relatively 
simple Drill-and-Practice strategy. Whereas DriU-and-Practice programs serve as an 
adjunct to the instructor, tutorial programs strive to replace the instructor as much as 
possible. Logical dialogue between the student and the program about specific 
problems is the hallmark of this strategy. These programs are typically capable of 
detecting not just simple calculations and data input errors, but also errors in reasoning 
by the student. Extensive feedback is provided to the student to assist in correcting 
these errors. [Ref 8] 

The tutorial strategy is most often viewed by the layman as the epitome of 
what a CAI should be. The vast majority of the early large scale endeavors in the field 
of CAI were of this type. As computer technology has advanced, improvement have 
been incorporated into the tutorial strategy and it remains the most preferred of CAI 
methods. This technique can be tailored to fit most any learning situation but it excels 
in imparting complicated conceptual skills. [Ref 9] 
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c. Problem Solving Tasks 

This strategy encompasses programs which are written to solve certain 
selected problems in a specific area. This is the least well defined of the five strategies 
presented in that a wide variance of the sophistication can exist between different 
programs within this strategy. This strategy places no boundaries on the various 
applications or level of difficulty of problems. In general, these programs tend to be 
more solution oriented and less concerned with developing computational or 
conceptual skills of the student. Most applications within this strategy parallel the use 
of a simple calculator; however, in this case, the programs have the ability to solve 
problems of far greater complexity. This strategy is well suited to quantitative courses 
which are primarily interested in the final solution and not the methods or techniques 
to derive the solutions. [Ref 10] 

d. Simulation 

This strategy is generally used for the generation and manipulation of data 
or the repetitive cycling of a model when environmental factors preclude normal 
methods. These environmental factors include limitations of time, money or 
equipments; and various safety considerations. 

This strategy is based on the mathematical manipulation of complex 
modules which places the student in a controlled real life situation. Most programs of 
this type allow the user to input a series of parameters and process these, in a 
compressed fashion, through the module. The resultant information is displayed for 
user perusal and may include pertinent comments to key the student to significant 
data. In general, no learning path is predefined. Instead, the student is allowed to learn 
through actual manipulation of these processes. These types of program are frequently 
used in statistical analysis and can be very effective learning tools. Concepts learned via 
this strategy are likely to be retained by the student longer. [Ref 11] 

e. Instructional Games 

The distinctive feature of this strategy is an attempt to use a student's 
competitive nature to achieve certain learning goals or skills. These programs most 
often pit the student against the computer in a win-lose situation in an effort to hold 
the student's attention and interest. Creative programming skills are required to make 
this strategy effective as a learning tool. This technique is ideally suited to elementary 
level education. 
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This thesis demonstrates Drill-and- Practice strategy. The Testing program 
is organized linearly with a brief introduction followed by the presentation of a series 
of questions in a domain of knowledge (field of aviation). The Testing makes use of 
Bayes' Theorem to adjust a lesson being presented to suit his/her understanding of the 
lesson. 

B. BAYESIAN ANALYSIS 

Bayesian analysis is concerned with the basic problem of assessing some 
underlying "state of nature" that is in some way uncertain. On the basis of whatever 
evidence does exist, some action or actions are to be chosen from among various 
possible alternatives. In some cases, the task of the investigator will be that of 
statistical inference, the only action to be taken being the acceptance or rejection of 
some hypothesis. In other cases, the actions will be the implementation of various 
possible decisions such as that the outcomes of these decisions depend upon the 
unknown state of nature. [Ref 12] 

The basis for Bayesian Analysis is the Bayes Theorem, this theorem provides a 
very powerful tool for statistical inference, especially when pooling informations from 
different source is appropriate. Thus, prior information resulting from economic theory 
and/or . from previous (real or hypothetical) samples can be combined with the 
information embodied in new observation, and this operation can be performed 
formally, within a rigorous mathematical framework. [Ref 13] 

Bayes' Theorem can be explained in simple terms as follows. Assume that there 
are two events A and B whose joint probability density function is given P{A,B). Then 



P(A/B) 



P(B/A) P(A) 

P(B/A)P(A) + P(B/A')P(A') 



(eqn 2.1) 



where A' represents the complement of the event A (that is, "not A") [Ref 14]. 

Notice that in the above equation there are two inputs under the form of 
probability density. The first input is the marginal probability density p(A) which 
defines the knowledge of event A prior to an observation. The second input is the 
probability density of the event B conditional upon the occurrence of event A, i.e., the 
probability that event B will occur given that event A has already occurred. This latter 
probability density provides an additional information of the event A in the form of 
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conditional probability density for event A under the assumption that event B had 
occurred, and the above equation provides a way in which to adjust the probability of 
the occurrence of event A in a rigorous manner. 




III. A BAYESIAN DIAGNOSTIC MODEL 



A. ASSUMPTIONS 

Assuming that the student has completed the study of the domain material and is 
ready to be tested upon his level of assimilation. At any point during a diagnostic 
session, the probability that the student wUl answer the remaining queries correctly 
depends on the proponion of the past queries on the same topic answered correctly. 

We further assume that the domain knowledge being imparted to the student is 
decomposable into blocks of concepts/ideas, so that levels of knowledge about various 
blocks of the domain can be independently measured. Blocks can be based on chapters 
outhned in textbooks. 

B, THE DIAGNOSTIC MODEL 

The proposed diagnostic model is illustrated in Figure 3.1. The model: 

1. Formulates hypotheses about a student's strengths and weaknesses regard to 
various concepts/ideas in a specific block of knowledge domain. 

2. Tests them using a set of carefully compiled questions. 

3. Derives conclusions about the student's problem areas. 

It has six steps to complete the above three functions. First, the hypothesis that 
the student is more likely an average student than either very bright or very poor is set. 
In the second step, a block of concepts/ideas is selected out of the domain that the 
learner is to be tested upon. A series of critical questions related to that block of 
concepts is then presented to the student. In the fourth step, as explained later in C, 
depending on the number of queries correctly answered, the hypothesis can either be 
rejected or not. If the hypothesis is not rejected, the process of querying the student 
with additional questions about the same block of concepts is repeated. On the other 
hand, if the hypothesis is rejected, we move to the next block for either of two reasons: 
(1) the student is far brighter, or (2) the student is far poorer than the average student. 
This avoids irritating the bright student by asking too many questions and at the same 
time avoids discouraging the poor student as well. At the end of the session, 
appropriate conclusions are made about the degree of proficiency the student, his 
strengths, weaknesses, misconceptions, and remedial strategies are suggested. 
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Figure 3.1 The Diagnostic Model 
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The power of the model lies in its ability to adapt a diagnostic session to the level 
of knowledge of a student automatically by limiting the type and numbers of questions. 
This is shown by the loops connecting step 5 to step 2 and step 3. If the initial part of 
a session strongly suggests that the student is knowledgeable about a certain concept, 
then the model skips the remaining part of the session and moves on to the next 
concept noting its conclusions on a global variable. The global variable forms a 
dynamic workspace and acts as a scratchpad. It enables the diagnostic model to create 
and store intermediate observations about the student performance. On the other hand, 
if the student is average, the entire session is offered. When all the concepts are 
exhausted, the Results component (global variable) is consulted and detailed diagnostic 
about the student's strengths, weaknesses, misconceptions, etc., is made and 
prescriptive measures suggested. [Ref 15] 

C: THE BAYESIAN APPROACH TO DIAGNOSTIC MODELING 

It was noted that the first and fourth steps of the diagnostic model comprised of 
formulating and testing hypothesis about the student's level of knowledge. Since there 
are looping between step4 and step3, step5 and step2, and step6 and step2, depending 
on the proficiency of the student, it is possible that the formulation and verification 
steps will be executed several times sequentially. While a student is tested upon a 
concept using short independent questions, it becomes necessary to update the 
hypothesis at the end of each cycle through the loop. We consider the Bayesian 
statistical techniques to be appropriate and viable for such sequential updating of 
hypothesis, since they can based on the set of the latest finding during a session, 
thereby exploiting the techniques as a means of operationalizing the diagnostic model. 

Before one can use the Bayesian approach, it is necessary to impart a few 
structural characteristics to the domain being tested upon the student. To achieve this, 
we shall proceed as follows. Let there be N blocks of concepts in the knowledge 
domain that are to be tested upon the student. Focus on the first block, say Cj . Let 
there be at most M questions related to Cj that may be posed to the student to test his 
expertise about the block. We shall divide the M questions into three groups, the first 
containing M ^ critical questions, the second containing M 2 complementary questions, 
and the third containing supplementary questions. The critical questions are those 
which are advanced in nature and cover all the important aspects of the block 
thoroughly (question 1 through 14 in Appendix A). In any diagnostic session with a 
student all the critical questions are always presented. If the student answers all of 
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them correctly, we presume the student need not to be subjected to the remaining (M- 
Mj) questions and is good enough to be tested upon the next block of concepts. The 
complementary questions are those which cover the non-critical yet important aspects of 
the block (question 15 through 30 in Appendix A). We presume that a close to perfect 
score in the critical questions combined with a perfect score in the complementary 
questions is once again an indicator of sufficient mastery over the block under 
consideration. The supplementary questions form the set of least advanced questions 
about the block (question 30 through 45 in Appendix A). They are repetitious in 
nature in that they do not cover any new aspects about the block. They merely split 
the critical and complementary questions into its elementary parts easy enough for the 
student to handle. The supplementary' questions are meant to be posed to the student 
only if the performance in the critical and complementary sections has been poor 
enough to warrant them. 

In introducing the Bayesian analysis into the above domain structure, we need to 
make assumptions about the prior probabilities that form the first set of estimates 
about the level of knowledge of a student in the knowledge domain, and a first guess of 
our initial degree of conviction about each of these assumptions. 

The following two strategies are suggested to generate the prior probabilities. In 
the first, we presume that there is no real way one can estimate the level of knowledge 
of a student beforehand and hence we postulate that the chance of the student being 
proficient or ignorant is fifty-fifty. Equal probability reflects complete uncertainty. We 
will refer to this hereafter as a fifty-fifty assumption. We shall denote this prior 
probability by Py = 0.50. The subscript u is chosen to indicate an uninformed opinion. 
In the second strategy, we assume that one way to estimate the level of knowledge of a 
student is to seek an estimate from an expert human instructor in the knowledge 
domain. Alternatively, it might also be determined by examining how the students 
have performed in similar tests historically. We shall denote this prior probability by 
using . The subscript ;c is chosen to indicate an expert opinion. Historically, if 
students have secured a grade B average in a particular knowledge domain, one can 
attribute a starting probability value of say, 0.80 for Pj^ . Further, we assume that our 
degree of conviction in the uninformed versus the expert opinion is a to b where a and 
b are weights. We can express this in probability notations as P(B^) = a/(a + b), and 
P(Bjj) = b/a + b, where P(B^^) denotes the strength of belief in the uninformed opinion 
and P(B^) that in the expert opinion. 
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Let Aj denotes the event of begining a diagnostic session by asking the student 
Mj questions about Cj and the number of questions correctly answered be Rj Before 
continuing the session uith the remaining (M-Mj) questions, we first revise our earlier 
opinions about the expert (i.e., P(B,^) ) in the light of the new evidence gathered from 
the diagnostic session so far (Aj) The mathematical formulation is shown below. 

/MA R, MpR, 

P(A,/B„) “L)<V (‘-Pu) (eqn3.1) 



/MA R, M,-R| 

PfAi/Bx) ■(^r )<Px) ‘ (>-Px> ' ‘ 

Applying Bayes Theorem, 

P(By/Aj) and P(Bj^/Aj) can be derived as follows: 



(eqn 3.2) 



P(Bu/Ai) = 



P(B^)P(A,/B^) 

P(B^)P(Ai/B^) + P(B,^)P(Ai/Bx) 



(eqn 3.3) 



and 



P(B^'A,) = 1-P(B^/Ai) (eqn 3.4) 

P(B^j/Aj) and P(B.^./Aj) represent the revised degree of confidence in the initial 
hypotheses given that the student had correctly answ’ered Rj out of Mj questions. 

In the light of the new findings, there are two possible options. One is to use a 
cut-off value for P(B,^,‘Aj), say alpha. Alpha is the minimum strength of belief in the 
expert opinion that is required to conclude that the student’s ability to do a test is 
good. Thus, if P(Bj./,Aj) > = alpha, we may conclude that the student is 
knowledgeable enough in Cj and not pose the remaining M-Mj questions. We can 
avoid exasperating a good student by asking far too many questions in a field he 
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appears to be well informed. We can store our conclusion about the student in the 
Results component and proceed with the remaining blocks of concepts in the domain 
in the same fashion. 

Conversely, if the cut-off criterion P(Bj^/Aj) > = alpha for a good student is not 
met, it leaves us with two other possible inferences about the student. One is that the 
student is just an average student as was anticipated under one of the assumption; the 
other possibility is that the student is far poorer. We may want to make a conclusion 
about which of these categories the student belongs before proceeding with the M-M j 
questions. In fact, we may choose to spare a poor student from these questions to not 
turn the enthusiasm off prematurely. To distinguish the poor student from the average 
one, we propose using a second cut-off criterion P(B^/Aj) < = beta. Beta is the 
minimum strength of belief in the expert opinion required to discern poor students. 
Thus, if P(B.^/Aj) turns out to be below beta, one might skip the remaining (M-M^) 
questions and move to next block. The conclusion that the student is poor about the 
concept block is stored in the blackboard for future reference. 

The other opinion arises if P(Bj^/A|) falls within both cut-off criterion. In this 
case, we continue questioning with the set of M 2 complementary questions and repeat 
the procedure. If the revised probability does not meet either criterion again, the 
session may be continued with the set of questions. On the completion of a student 
session, by assessing the remarks stored on the Results component from time to time 
during the session, detailed diagnosis can be made about the level of proficiency of the 
student and intimated to him along with remedial suggestions. [Ref 16] 

D. EXAMPLE OF B RIGHT-STUD ENT PERFORMANCE 

Let us assume that after a detailed analysis of the knowledge domain, an expert 
instructor in the domain came up with 25 (i.e., M = 25) questions about the various 
aspects of the block. Let these 25 questions consist of 5 critical (Mj = 5), 8 
complementary (M 2 = 8 ), and 12 supplementary (M 3 = 12) questions. Let two of the 
starting assumptions be: ( 1 ) the odds are even a student is knowledgeable or ignorant 
about the block of domain, or P^ = 0.50; (2) since historically students in the domain 
have done better, it is more likely P^^ > P^^ hence say P^ = 0.70. Further, we shall 
assume that our level of confidence in these two assumptions (p^ and p^) is 1 to 4. 
We can express this in probability notations as, P(B^) = 0.20 and, P(Bj^) = 0.80. We 
shall further assume the critical cut criteria, alpha and beta are 0.90 and 0.40 
respectively. The following scenario demonstrates some practical implications of the 
proposed diagnostic model. 
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Suppose that when the student was asked the critical set of 5 questions, he 
answered 4 correctly. Let Aj denotes this event. We have now’ to make a decision as to 
whether the student is bright or poor enough based on this performance and whether 
or not w'e should continue with the remaining 20 questions at all. To do this, we first 
compute P( A By) and P(Aj/Bj^) using equations 3.1 and 3.2. 



P(Ap'By) 



5 \ 4 5-4 

^ j (.5) (1-5) = 0.156 



(eqn 3.5) 



P(Ai/B,^) 




0.360 



(eqn 3.6) 



From our initial assumption about the strength of belief in the uninformed and 



expert opinions, w'e have 




P(By) = 0.20 


(eqn 3.7) 


P(B^) = 0.80 


(eqn 3.8) 



Applying the results of (3.5), (3.6), (3.7), and (3.S) in equations (3.3) and (3.4), 
we get P(By'Aj) = 0.09S and PIB^^/Aj) = 0.902. 

Comparing P(Bj^/Aj) = 0.902, w'ith the criterion alpha = 0.90, w’e reject the 
assumption that the student is average. One can also note that our level of confidence 
between P(By) and P(B^^) has improved from 1 to 4 to 1 to 9.20 in the light of the 
session with the students. The conclusion here is that the student is above average. In 
this case w’e suspend asking further questions about the block of concept in 
consideration to the student, and proceed directly to the next block of concepts. 

The fact that student was well know’ledgeable about a block of domain is also 
noted in the Results component so that at the end of -the entire diagnostic session, 
consolidated comments about the overall level of proficiency of the student in the 
know’ledge domain may be made. 
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E. DEVELOPMENT OF QUESTIONS FOR THE TESTING SYSTEM 

In order to demonstrate how the proposed system of this thesis works, the 
questions used in this study are drawn from "Student Pilot's Flight Manual" [Ref 17]. 
This book is arranged into five chapters; (1) Parti- Before The Flight; (2) Part2- Pre- 
Solo; (3) Part3- Post-Solo Maneuvers; (4) Part4- Cross-Country and Night Flying; (5) 
Part5- The Written and Flight Test [Ref 2: p.5]. For purposes of testing the utility of 
the system with question sets, only "Parti- Before The Flight" was utilized and divided 
into seven ideas; (1) Starting to fly; (2) The Airplane and How It Flies; (3) Cockpit- 
Instrument and Systems; (4) Preflight-Check; (5) Starting the Airplane; (6) Taxiing; and 
(7) Pre-Take-off and Cockpit Check. It should be noted that the other questions sets 
could be easily developed from other chapters or other texts for basic and advanced 
flight training. The Testing System demonstrated here may also be applied to other 
subject areas in which self-testing and self-evaluation is desirable. 

Forty-five questions are extracted from the above text book. According to the 
Bayesian Diagnostic model explained in Chapter 3 of this document, these questions 
are separated into three sets. The first set consists of 14 questions which are considered 
critical to the understanding of the chapter. The second and the third sets consist of 16 
complementary questions and 15 supplementary questions respectively. The first and 
the second sets of questions cover all important aspects about the chapter and the 
third set of questions does not cover new aspects about the chapter but reinforces 
already introduced concepts. However, the proposed model explains this concept in 
more details. 

The forty-five questions are grouped into three types, procedural, calculative, and 
physical questions. Each question is ranked as an easy or difficult concept, which can 
be determined by the system. The purpose of this technique permits the Testing 
System to analyze in detail the specific level of the student's difficulty in each testing 
area, and provides more precise guidance for student improvement. The questions are 
presented in Appendix A. 
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IV. TESTING: A PROTOTYPE OF PILOT KNOWLEDGE EVALUATION 



A. TESTING SYSTEM ARCHITECTURE 

The Testing System is a CAI tutorial system with a diagnostic monitor that 
makes use of the Bayesian model described in Chapter 3. Testing is implemented in the 
knowledge domain that forms part of study in the field of aviation. The architecture of 
Testing is shown in Figure 4.1. The Testing System consists of four components: (1) 
Inference Engine; (2) Data Base; (3) Temporary Results; (4) Historical Data. 




Figure 4.1 Testing System Architecture 

The Inference Engine holds information about the concepts in a domain of 
knowledge(field of aviation). It also contains the control strategies to access the other 
components in the system as well as the student user. Above all, it accommodates the 
Diagnostic iMonitor that contains the proposed Bayesian Model. 



23 



The Data Base consists of a bank of critical, complementar>', and supplementary’ 
questions on the various concepts represented in the domain of knowledge (field of 
aviation) for this prototype. Examples of such questions are shown in Appendix A. 

The Temporary Results component stores results after each question and 
essentially acts as a scratch-pad. The results are discarded once you exit the program. 
The Historical Data component holds any data about the student's performance in 
each test that are needed to be kept for future use. With this component, Testing can 
determine if the student's performance is better or worse than the previous one. 

B. DATA STRUCTURES 

For efficient operation the program should not be spending a lot of time to 
search for the data needed to be presented to the student. The required questions need 
to reside in memory in a data structure that allow quick access. The method by which 
books/concepts and ideas are numbered is also important. 

In this thesis, a logical tree structure was used to store the questions bank as 
shown in Figure 4.2. It is implemented with the use of menus. 
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The root node represents the domain of knowledge (field of aviation). Figure 4.3 
represents the root node. 



WELCOME TO THE FIELD OF AVIATION 



Figure 4.3 The Representation of Root Node 

From the root, if the student enters the command to continue ("press any key to 
continue"), he/she is going to the next level which is book/concept level. The menu of 
book/concept is then presented (Figure 4.4). 



PLEASE SELECT ONE OF THE FOLLOWING 


1., 


....STUDENT PILOTS FLIGHT MANUAL 


2.. 


....PRIVATE PILOT STUDY GUIDE 


J., 


....INSTRUMENT FLIGHT RULES 


0. 


....for " QUIT " (exit program ) 




( type only number you want ) 



Figure 4.4 Menu of Valid Concepts in The System 

The user/student can choose one of these nodes which is the book/concept number. 
After the user/student selects a number (node), he/she is going to the next level of 
chapter, and the menu of chapter is presented (Figure 4.5). 

The user/student may again choose one of the nodes which is the chapter number. 
Each node (number) in this level represents chapter of the selected book which 
contains test questions. We see that. The number of books is the number of children of 
the root node (field of aviation). The book/concept is the parent of the chapters and 
the number of chapters is the number of children of that book. By using tree structure, 
choosing any chapter to test is like traversing downward the tree from the root. The 
Testing program handles a domain of knowledge (field of aviation) with 3 concepts and 
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" STUDENT PILOT FLIGHT MANUAL " 
SELECT ONE OF THE FOLLOWING 

TYPE 1 for PART - ONE 

TYPE 2 for PART - TWO 

TYPE 3 for PART - THREE 

TPYE 0 for SELECTING ANOTHER CONCEPT 



Figure 4.5 Menu of Valid Chapters 

3 chapters in each concept for an illustration of the logical tree structure. It was 
decided to use text files to store questions; each file contains questions, correct 
answers, categories of questions and comments for each test (see Figure 4.6 for 
explanation). 

In order to store the student's performance, an array of records is used, and 
stored in main memory by defining it as a global variable. One record per chapter/test 
contains all student's performance about that test. It contains fields for how many 
questions in the test and, in each category of question, the number of correct and 
incorrect answered, number of questions completed, the names of concepts and ideas, 
and also detailed performance in each category of questions ( see record Chap in 
Appendix E ). The variable Chapter (see Appendix E) is of type array. Each element 
of the array contains one record of student's performance to be used by the program to 
diagnose the student according to the proposed model and to provide scoring and 
commentary at the end of the session. 

C. TESTING SYSTEM CONFIGURATION AND DESCRIPTION 

The test subject chosen was Pilot Knowledge Evaluation. The program Testing 
is written in Waterloo Pascal on the IBM mainframe (IBM 3033 AP). The program as 
presented is designed for a pilot or student pilot. The student/user is assumed to have 
already learned the basics through traditional classroom lecture techniques or by 
himself and now needs to test his/her knowledge. Questions about various concepts 
are stored in the Data Base component so that the student/user can select any concepts 
which he/ she has already learned and be evaluated as to whether or not he/she has 
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absorbed the instructional material as well as to be diagnosed as to where he/she has 
difficulties. The user is only required to know the minimal knowledge of how to 
operate a computer terminal. 

As the student is going through the test, his/her performance will be recorded in 
the Temporary Results component, and (after a certain amount of questions have been 
asked which are required by the control strategy), the system will determine the 
probability of his/her ability to answer the rest of the questions. If this probability/ P) 
exceeds the upper cut-off criterion (A = 0.9), he/she is assumed to have mastered the 
lesson. He/she is exempted from finishing that lesson, and may continue onto the next. 
If the probability is below the lower cut-off criterion (B = 0.4), he/she is deemed to 
have insufficient knowledge about the lesson. The control strategy will either suggest 
the student restart that lesson again or select another concept to study. If P is in 
between the two cut-off criteria, he/she is assumed to be average, thus, further 
questioning is required. At the end of the session, when the student wants to exit the 
program, appropriate conclusions are made about the degree of proficiency of the 
student, his/her strengths, weakness, and misconceptions, and remedial strategies are 
suggested. Appendix B and C demonstrate Testing. 

D. TESTING SYSTEM DETAILED DESIGN AND DESCRIPTION 

1. Design of the File 

The question set for each test is designed as a separate text file. When a 
student selects a test, a specific text file contains only questions for that test is called. 
As explained early in Chapter 3 (Development of questions for Testing System), the 
questions are: (1) separated into 3 sets: critical, complementary and supplementary 
questions: (2) grouped into 3 types: procedural, calculative and physical questions; (3) 
determined individually by the system as an easy or difficult question. These questions 
cover seven aspects of "Parti- Before The Flight", in "Student Pilot's Flight Manual" 
[Ref 18]. The design of this file is shown in Figure 4.6. 

In Figure 4.6, the line under the correct answer contains one number and two 
characters. The first digit indicates (for the above question) which of the seven areas 
the question refers to (Parti- Before The Flight}. Codes for these areas are shown in 
Figure 4.7. The character following the first number identifies the question type, 
procedural, or calculative, or physical question. Codes for these types are shown in 
Figure 4.8. The last character on the line under correct answer indicates the above 
question is considered an easy or difficult one ("e" stands for an easy question, "x" 
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File : Filel Text A1 

45 (* number of questions in this file *) 

14 (* number of critical questions *) 

16 (* number of complementary questions *) 

15 (* number of supplementary questions *) 

1) (* Body of question *) 



1 

2 (* Four multiple choices *) .... 

3 

4 

S (* symbol indicates end of questionl *) 

2 (* correct answer *) 

2se (* to be explained *) 

...(* comment & suggestion for question I no more than one line *)... 

2 ) 



1 

2 

3 

4 

S 

2 

3px 

...(* comment & suggestion for question 2 no more than one line *)... 
3) • 



Figure 4.6 The Design of File Containing Questions 
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stands for a difficult question). For example, "2se" on the line under correct answer in 
question 1 of Figure 4.6 means the above question is in the area of "The Airplane and 
How It Flies" (2), it is a physical question (s), and considered an easy one (e). An 
example of this file and questions are shown in Appendix A. 



Codes # 


Meaning 


"1" 


" Starting to Fly " 


”T 


" The Airplane and How It Flies " 


"3" 


" Cockpit- Instrument and System " 


"4" 


" Preflight Check " 


"5" 


" Starting The Airplane " 


"6" 


" Taxiing " 




" Pre Take-off and Cockpit Check " 



Figure 4,7 Codes for Areas in Part 1- Before The Flight 



Codes # 


Meaning 


// // 
P 


" Procedural question " 


"c" 


" Calculative question " 


// // 
s 


" Physical question " 



Figure 4.8 Codes for Types of Questions 

2. Description of Operation and Examples 

The proposed program was designed using a top-down stepwise refinement 
methodology. Figure 4.9 shows the steps of operation in Testing. 

The operation of Testing begins with a students need to test his/her knowledge 
of a concept which is provided in the domain of knowledge (field of aviation prepared 
in the system). The student is provided a listing of concepts, which gives him/her a list 
of valid ideas provided in the system. Figure 4.4 shows the menu of concepts. 



29 




student's abiiitv 
to answer questions 
A = Upper cut-ofT criterion = 0.9 
B = Lower cut-ofT criterion = 0.4 



Figure 4.9 Testing System's Flow Chart 



30 



Once the student entered the desired concept to test upon, information about 
the test's processes is presented. This gives the student some idea of what is going to 
be presented during the session. When the student enters the command to continue 
("press any key to continue"), the list of chapters within his/her selected concept is 
provided so he/she may choose the one that he/she wants to work on. A sample list is 
shown in Figure 4.5. 

The very last step of the system is presentation of the fmal score and 
comments which is made after the student enters the quit command ("type.. 0"). This 
shows the student's performance in the test (mistakes that he/she made and percent 
grade in every aspect of the selected chapter). If the percent grade in any aspect is less 
than 50, the system gives notice to the student of his/her mistakes and areas of 
difficulty. The system also records the student's performance permanently in a 
historical file for future use (student ID#, name, date, and grade in every test). 
Previous historical data is provided so that the student wUl know his/her status to date. 
Examples of score and commentary are shown in Appendix B and C. Appendix B 
demonstrates the performance of a bright student. Appendix C demonstrates the 
performance of a weak student. 
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V. CONCLUSIONS 



A. DISCUSSION OF THE RESEARCH 

In this thesis, a modification of the standard CAI system is presented. This 
modification is unique in that it combines the standard tutorial strategy of CAI and the 
Bayesian Analysis to tailor a lesson to better suit the student's ability or proficiency in 
mastering the subject being taught. It demonstrated the feasibility and benefits of 
using an intelligent CAI system to assist learning strategy in any domain of knowledge. 
The use of Pascal as a practical language for the intelligent CAI systems was shown. 
The benefits of Testing are: 

• Provides a simple approach. 

• Easy to develop and implement. 

• Suited to courses where a repetitive practice is required to learn a skill or a 
concept. 

• Allows a student to work through a large number of problems to gain an 
intuitive understanding of the solutions procedures. 

• Uses little core memory, runs fast, and can be used in portable microcomputer. 

• Provides self-testing or self-evaluating. 

• Reduces the instruction time of bright students. 

• Provides the slow ones to progress along at their own pace. 

• Accelerates the learning process through identification of weak areas. 

B. TESTING SYSTEM LIMITATIONS 

Testing is a prototype program that was designed to demonstrate the concept 
that an intelligent CAI system using the Bayesian Diagnostic model could provide 
valuable assistance in tutoring the student to learn the lesson. This prototype {program 
demonstrates concepts in the field of aviation in which the author has experience. 

Due to time limitations, the Testing program was developed using a limited 
number of concepts and test questions. One test question set has been made from 
Chapter 1 of "Student Pilot's Flight Manual" [Ref 19]. The program was designed to 
handle 3 books (see Figure 4.4) with 3 tests in each book. With this designed, it was 
possible to demonstrate the functions of the proposed model. Since this program was 
written in Waterloo Pascal on the IBM mainframe (IBM 3033 AP), it can be used in 
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some personal computers, if the program is rewritten so that it can be used on more 
personal computers. It will be more practical for some small organizations to use this 
syetem. Another limitation is this program is not graphics-interfaced which would help 
the user/student in learning. The system could display pictures to describe some ideas 
that cannot be done well by text. 

C. FUTURE PROGRAM IMPROVEMENTS 

The purpose of this document is to demonstrate the proposed model in Chapter 
3. With the time limitations, the program handles only a few concepts. The data 
structures used in the program are arrays, records, and files. The program can be easily 
modified to handle many more concepts by increasing number of arrays to store the 
records of student performance. It is easy to increase the arrays of records by changing 
the global constants such as maxbook, and maxpart (see Appendix E) that indicate 
maximum number of concepts and maximum number of tests in a concept respectively. 
The Testing program is composed of only a few modules or procedures. It is easy for 
an experienced programmer to understand the functions of each module. It would take 
only a few days to develop Testing program to handle more concepts. The major tasks 
are to do some changes in those procedures that determine which concept is selected 
by the user/student and present him/her with the test questions. The procedure 
Booklist, Beginsession, Whatchapter, and Selectchapter must be modified by adding 
numbers that identify the concepts in the same fashion as previously done (see 
Appendix E). The only time-consuming task is to develop the questions for the 
concepts that are needed. Time to complete the questions is dependent on how many 
concepts are required and the people who develop the questions. 

For better program improvements, a linked-list representation is recommended to 
store the student's performance instead of arrays. One advantage of a linked-list 
representation over the arrays used in Testing is that less memory space can be used. 
For better tutoring, a graphics interface is recommended. 
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APPENDIX A 

ILLUSTRATION OF QUESTIONS USED TO DEMONSTRATE TESTING 

SYSTEM 

The following questions are drawn from Student's Pilot Flight Manual. 



45 

14 
16 

15 

1) Which one of the following statements is correct concerning "torque" 

corrections (U.S. -built engines)? 

1 In a climb, left rudder is necessary to keep the airplane 

straight. 

2 In a climb, right rudder is necessary to keep the airplane 

straight . 

3 In a dive, right rudder is necessary to keep the airplane 

straight. 

4 When power is added abruptly, the airplane will tend to 

roll to the right. 

$ 

2 

2se 

'Torque' see chapter 2 ,fig. 2-9, SPFM. 

2) Which one of the following statements concerning the Four Forces 

is correct? 

1 In straight and level flight, all Four Forces have 

equal values. 

2 In a climb, lift is greater than weight. 

3 In straight and level unaccelerated flight, thrust is 

greater than drag. 

4 Lift acts perpendicular to the relative wind, whether 

the airplane is climbing, flying, flying straight and 
level, or gliding. 

4 

2se 

'Four Forces' See chapter 2, fig. 2-3, SPFM. 

3) True altitude is height above the sea level, and absolute altitude 

is height above the surface. With this in mind, choose the correct 
answer to the following: An airplane is flying over the ocean at 
an actual height of 10,000 feet over the water 
(altimeter setting' and temperature unknown). It is flying at its 

1 density and absolute altitudes. 

2 pressure and absolute altitudes. 

3 true and absolute altitudes. 

4 true and density altitudes. 

$ 

3 

3px 

'Study all four types of altitude', in chapter 3. 

4) You are flying at an indicated airspeed of 120 knots at 6,000 feet 

(assume no instrument error) the outside air temperature is +3 
degrees Celsius. Your computer is back on the counter at the airport, 
but using rule of thumb you can estimate that your true airspeed 
is approximately 

1 140 mph. 

2 134 knots. 

3 124 mph. 

4 144 knots. 

$ 

2 

3cx 

see chapter 3, 'Rule of Thumb', page 134, SPSG. 
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5) If you are referring to the magnetic compass on a heading of east, 
an acceleration of airspeed(maintaining a constant altitude) 
will result in 

1 a more northerly indication. 

2 a more southerly indication. 

3 no change in compass indication, since it is correct on 

east or west heading. 

4 northerly turning error. 



1 

3sk 

'ANDS', p.45 (SPFM) and p. 134, #308 (SPSG). 

6) During the pretake-off check after checking the mags, you pull 
the carburetor heat ON at 1800 rpm(or the manufacturer's 
recommended number). There is no change of rpm (plus or minus), 
when you pull it ON or for 30 seconds thereafter. This is a sign 

1 carburetor ice was present before the heat was applied. 

2 there was no carburetor ice present before the heat was 

applied. 

3 the carburetor is not getting heat from the system. 

4 you did leave the heat on long enough to get an effect. 

$ 

3 



4px 

'Function of carburetor', see item G in Fig. 4-M,Fig. 4-2 in SPFM. 

7) After starting your airplane's engine on a cold morninq, you taxi out 

and make your pretake-off check. After the mag and carburetor heat 
check you reduce the power to idle and find that even though the 
engine is warmed up it lopes and run very rough, nearly quitting but 
not quite. A likely problem is that the 

1 fuel selector valve has been in the OFF position since 

before starting. 

2 cabin heat is ON. 

3 primer is not in and locked. 

4 electric fuel pump is not ON. 

$ . 

3 

S sx 

'Primer system'. Fig. 23-17, SPFM. 

8) It is a very hot day and the trainer you are going to use has just 
come down from a flight. After thorough preflight check you get 
fastened in. After using your usual procedures you holler CLEAR 
and hit the starter; the propeller turn over and over with no 
indication that the engine wants to start. This is probably because 

1 the mixture is not rich enough. 

2 the engine is loaded( flooded) . 

3 the carburetor heat is not ON. 

4 the engine has cooled too rapidly. 

2 

5se 

'Hot start procedure', in chapter 5, SPFM. 

9) You hit the starter of your trainer, and as the propeller turns over 

(the engine hasn't started running well yet), you note that 
spectators are starting wide-eyed at the bottom area of the cowling. 
Your initial best move is to 

.1 open the window and state firmly that it is impolite to 

stare. 

2 pump the throttle to help assure that plenty of fuel is 

getting to the carburetor. 

3 keep the engine turning over and turn off the mixture 

and fuel. 

4 stop the starting procedure until they stop staring. 

$ 

3 

5se 

see Pilot Operation Handbook, 'starting procedure'. 

10) Hold lines are a combination of two solid and two dashed lines 

crossing the taxiway before it intercepts a runway. The taxi 
guidance line extends through the hold lines and onto the runway. 
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This means that 

1 you don't have to slow up before taxiing onto the 

runway at uncontrolled airports. 

2 you only to recognize hold lines at controlled airports 

because ground control will give or refuse clearance to 
cross them. 

3 hold lines should be recognized at all airports. You 

will need ATC clearance to cross them at controlled airport 
at uncontrolled airports you should always hold and check 
for traffic before going onto the runway. 

4 the taxi line has precedence over the hold lines at all 

airports. 

$ 

3 

6px 

'read chapter 25 of SPSG, Flying the cross country'. 

11) You are taxiing a tricycle-gear airplane with a strong wind from 

the right rear. The control positions should be as follows; 

1 Control wheel forward and turned right, rudder as 

necessary to keep the airplane straight. 

2 Control wheel forward and turned left, rudder as necessary 

to keep the airplane straight. 

3 Control wheel aft and turned left, full left rudder. 

4 Control wheel forward and turned left, full right rudder. 

$ 

2 

6sx 

'study technigue of taxiing in a strong wind.' 

12) When taxi downwind with strong wind, you should; 

1 hold the wheel forward so that the wind strike the down 

elevator and keeps the tail from rising. 

2 hold the wheel back so that the wind force the propwash 

hold the tail down. 

3 taxi with engine idle, use brake to keep speed down 

because of the wind forces the airplane to be fast. 

4 taxi with small power and use brake to keep speed. 

1 

6sx 

'How to taxi downwind with strong wind', see SPFM, p.37 

13) You are checking the magnetos at the run-up spot and inadvertently 

turn the switch to OFF. The best move for you before turning it 
back on is to 

1 pull the carburetor heat ON. 

2 close the throttle. 

3 pull the mixture to idle cut-off. 

4 switch tanks. 

S 

2 

7px 

'Accidentally turning magnetos switch of during run-up' . 

14) You have just completed the run-up. Just before starting to taxi 

into the take-off position, you realize that the fuel gage shows 
the tank you ran up on is very low in fuel while the other is 
nearly three quarters full. You recall this was that the case when 
you visually checked the tanks, but you forgot to put the selector 
on the fullest tanks. You should 

1 switch tanks and continue the take-off. 

2 keep the airplane on the tank you did the run-up on and 

go ahead and take-off. 

3 switch tanks and run the engine up to at lease 1500 rpm 

for a minute or more, or redo the mag and carburetor heat 
check. 

4 do none of the above. 

$ 

3 

7px 

'Very low fuel gage before taking-off . 

15) Ypu are solo and just lifting off (with the runway gone behind) 

it sounds as if the engine is banging itself to destruction. The 
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airplane is performing as usual (rpm, pressures , and temperatures 
are okay) . The best move in this case is to 

1 maintain control of the airplane as you continue to climb. 

2 land straight ahead immediately. 

3 turn back to the airport without delay. 

4 start dumping ballast overboard. 

$ 

1 



7px 

'Rough engine right after taking-off . 

16) Which of the following combinations would most likely cause loss of 

power and/or a rough-running engine during the pretake-off check? 

1 Airport at a high elevation, hign outside air temperature, 

mixture full rich, and carburetor heat ON. 

2 Airport at sea level, low outside air temperature, 

mixture full rich, and carburetor heat ON. 

3 Airport at a high elevation, high outside air temperature, 

mixture leaned, and carburetor heat OFF. 

4 Airport at sea level, high outside air temperature, 

mixture full rich, and carburetor heat OFF. 

$ 

1 

7sx 

'causes of losing power and/or rough engine in pre- takeoff 

17) You are a private pilot and plan to fly a trainer into a 2000-foot 

farm strip. You check the Pilot's Operating Hand-book and see that 
the airplane can land or take-off with about 500 feet to spare under 
the existing conditions. At the end of the pre-takeoff check you 
close the throttle and the engine idles at 800 instead of the 
recommended 600 rpm. You should 

1 continue the flight. 

2 taxi back and get the mechanic to reset the idle to the 

recommended value before flying. 

3 continue the flight but expect a steeper approach and short 

landing roll because the extra windmilling effects will 
create drag. 

4 be sure to notify an instructor or mechanic about the high 

idle value after you get back to the home airport. 

2 



‘idle rpm exceeded at the end of pre-takeoff check'. 

18) Taking off at higher altitudes and temperatures for a particular 
airplane (at a given weight) will require 

1 more runway because the air is less dense. 

2 less runway because the air is less dense. 

3 the same amount of runway because the less dense air 

create less drag, making up for any power loss. 

4 more runway because the air becomes more dense with 

increase in altitude. What horse power is required to raise 
55 pounds to a height of 6 feet in 3 seconds? 



1 

2sx 

'taking off at high altitude and temperature, p.6 SPFM' . 

19) What horse power is required to raise 55 pounds to a height of 
6 feet in 3 seconds? 

1 1 / 10 . 

2 1/50 

3 1/5. 



$ 

3 

2ce 

'how to determine horse power at a situation'. 

20) The drag resulting from lift being produced is 

1 parasite drag. 

2 interference . 

3 induced. 

4 gyroscopic. 
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$ 

3 

2sk 

'DRAG', and see Fig. 2-14, 2-16 in SPFM. 

21) Which of the following statements is correct concerning drag? 

1 Induced drag increases as the airspeed increases. 

2 If parasite drag is 200 pounds at a straight adn level 

speed of 100 knots, at 200 knots it will be 800 pounds. 

3 Skin friction drag increases with increased airspeed. 

4 Induced drag decreases as the airspeed decreases. 

$ 

2 

2ck 

'study the definition of DRAGS in SPFM.' 

22) You are flying on a solo cross-country in a trainer and realize 

that the oil pressure gage is at ZERO. Your best initial move 
would be to 

1 land immediately; the engine will stop within 3 minutes. 

2 watch the oil temperature gage for an increase in 

temperature as you fly toward an airport or better terrain. 

3 pay no attention to it but continue your flight. 

4 land immediately to 7700 on the transponder 

(if you have one) and call MAYDAY on 121.5. 

$ 

2 

3px 

'emergency procedure for zero oil pressure in flight.' 

23) If the engine-driven vacuum pump fails in the average trainer, 

you would e^ect to lose the use of 

1 airspeed and altitude indicators. 

2 altimeter and altitude indicators. 

3 altitude indicator and heading( direction) indicator . 

4 vertical speed indicator and turn coordinator. 

$ 

3 

3sx 

'loss of engine-driven vacuum pump, see Fig. 3-22, SPFM.' 

24) At a calibrated airspeed of 150 knots at a density altitude of 

10,000 feet, you want to set up a standard rate turn. Using the 
altitude indicator for bank angle, you would set up a bank of 
approximately 

1 15 degrees 

2 25 degrees 

3 35 degrees 

4 10 degrees 

2 

3cx 

'standard rate turn at any particular airspeeds.' 

25) Which one of the following items would require the most 

electrical current when in operation? 

1 Landing light. 

2 Turn and slip indicator. 

3 Flashing beacon. 

^ 4 Navigation receiver. 

1 

3se 

'electrical system.' 

26) Which_ of the following would most likely result in a sudden increase 

in oil consumption in the engine of an airplane? 

(There are no outside leaJcs in evidence.) 

1 Eroded spark plug electrodes. 

2 Worn or broken piston rings. 

3.. ... A poor adjusted carburetor. 

4.. ... A coagulated frammis. 

$ 

2 

4sx 

'sudden increase in oil consiomption. ' 
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27) As an all-around procedure you should 

1 deliberately run a tank dry to use all the fuel only if it 

is a fuel injection system. 

2 ej^ect a delay of 1-2 seconds in power after the selector 

is switched from an empty to a usable tank with the fuel 
injection system. 

3 deliberately run a tank dry only at altitude above 1500 • 

feet MSL 

4 never deliberately run a tank dry. 

4 

4px 

'deliberately run a tank dry during flight?' 

28) You have your private certificate ana are taking a friend for a 

first ride. You find that the battery is dead. It's late and 
there is no one else at the airport, but jumper cables are 
available. Also, you have checked out to nand-crank the airplane. 
Your next move would be to 

1 carefully show the friend how to hold the brakes and turn 

the ignition switch ON while you hand-crank the airplane. 

2 show the friend how to hold the brakes and use the starter 

while you set up jumper cables from your car. 

3 Stay in the airplane to start it, and the friend can work 

the jumoer cables (revving up the car and then removing 
the cables after the start). 

4 cancel the flight until the airplane's electrical system 

is back to normal. 

$ 

4 

5px 

'procedure for dead battery before taking off.' 

29) A soft or spongy brake is usually the result of having air in the 

system. To get braking action you should do which of the following? 

1 Pump the braxe pedals. 

2 Hold the pressure but do not pump. 

3 Pull the parking brake handle outward. 

4 Stay completely off the brake. 

1 

6px 

'procedure for having a soft or spongy brake while taxiing.' 

30) The "best" fuel-to-air ratio is approximately 

1 1 to 15. 

2 15 to 1. 

3 1 to 7. 

4 20 to 1. 

$ 

1 

5px 

'fuel-to-air ratio.' 

31) The angle between the relative wind and the chord line of the 

airfoil is the 

1 angle of incidence. 

2 angle of attack. 

3 planning angle. 

4 angle ot climb or glide. 

2 

2pe 

'angle of attack', see Fig. 2-5, SPFM. 

32) Brake horsepower is 

1 Horsepower being developed at the crankshaft. 

2 Horsepower developed by the propeller in moving the 

airplane through the air. 

3 measured as pounds of thrust horsepower in older airplane. 

4 always the same as thrust horsepower in newer airplane. 

1 

2pe 

'brake horse power', p.6, SPFM. 
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33) Gravity always acts 

1 opposite to weight. 

2 parallel but opposite to lift. 

3 perpendicular to the total drag. 

4 toward the center of the earth. 

$ 

4 

2pe 

'gravity, see "Weight", SPFM. ' 

34) In computing airplane performance, the pilot or engineer uses 

1 density altitude. 

2 true altitude. 

3 absolute altitude. 

4 indicated altitude. 

$ 

1 

3ce 

'computing airplane performance.' 

35) Indicated airspeed corrected for instrument and position error is 

1 true airspeed. 

2 calibrated airspeed. 

3 equivalent airspeed. 

4 static pressure. 

$ 

2 

3ce 

'understanding all four kinds of airspeed.' 

36) Equivalent airspeed corrected for density altitude effects is 

1 calibrated airspeed. 

2 indicated airspeed. 

3 true airspeed. 

4 density airspeed. 

3 

3ce 

'the relations among the airspeeds.' 

37) The job of pitot tube is to admit what pressure or pressures to the 

airspeed indicator? 

1 Dynamic only. 

2 Static only. 

3 Static and vertical changes. 

4 Dynamic and static. 

4 

3pe 

'the function of Pitot tube,' see POH. 

33) A job of the alternater is to 

1 provide a spark to each magneto as needed. 

2 provide vacuum for the gyro instruments. 

3 run the electrical components and keep the battery charged. 

4 alternate the power from the brake system to the electrical 

system. 

$ 

3 

3pe 

'the functions of alternater, check POH.' 

39) The majority of turn and slip indicators or coordinators in current 

smaller airplanes are powered by the 

1 static pressure system. 

2 engine-driven vacuum pump. 

3 hydraulic system. 

4 electrical system. 

4 

3pe 

' soures power of instrument indicators.' 

40) The vertical speed indicator operates on the principle of 

1 change in outside atmospheric pressure. 

2. ... .absolute atmospheric pressure. 

3 precession. 
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4 dynamic pressure from the pitot tube. 

1 

3pe 

'how the vertical speed works.' 

41) The addition of carburetor heat 

1 leans the mixture. 

2 has no effect on the mixture. 

3 riches the mixture. 

4 cause oil pressure fluctuations. 

$ 

3 

4se 

'understand the uses of carburetor heat.' 

42) An advantage in the fuel injection system over the carburetor is 

that it 

1 has better fuel distribution to the cylinders. 

2 does not require an alternate air system. 

3 can be used best with fixed-pitch propellers. 

4 has a simpler oil system. 

1 

4pe 

'the advantages of fuel injection system.' 

43) Smoking in an airplane 

1 is useful in finding out the wind direction by blowing 

the smoke out of the window. 

2 may cause you to be thrown overboard by a nonsmoker. 

3 can cause clogging of the vacuum system air filter. 

4 can clog the carburetor heat. 

$ 

3 

4se 

'causes of carburetor clogged during flight.' 

44) You start the engine ana check the oil pressure gage. The needle 

doesn't move right away, but you figure that since it's summer 
you can expect it to come to normal operation within 

1 42 hours and 26 minutes. 

2 5 seconds. 

3 30 minutes. 

4 30 seconds. 

$ 

4 

5pe 

'operating of oil pressure indicator after starting the engine.' 

45) Taxiway marking are 

1 white dashed lines. 

2 yellow solid lines. 

3 yellow dashed lines. 

4 white solid lines. 

$ 

2 

6se 

'Taxi Marking', see SPSG P.19 
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APPENDIX B 

DEMONSTRATION OF BRIGHT-STUDENT PERFORMANCE 
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-k k 

k k 

^ WELCOME TO THE FIELD OF AVIATION * 

k k 

k k 

kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk 



please type your number 
your name ? 

today date, (ex. 10 Apr 87) 
"press enter to continue" 



kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk 
k k 

* PLEASE SELECT ONE OF THE FOLLOWING * 

k k 

* 1 STUDENT PILOTS FLIGHT MANUAL * 

* * 

* 2 PRIVATE PILOT STUDY GUIDE * 

* * 

* 3 INSTRUMENT FLIGHT RULES * 

■k k 

* 0 for " QUIT " (exit program) * 

k k 

* (Type only number you want ) * 

k k 

kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk 
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In each test which you will work on 

I There are 3 sets of questions. 

\ Each set is different in level of difficulty 

I The most difficult set will be presented 
first and then the next two sets. 

I You will be evaluated after completing each 
of the first two set of questions. 

I If you are about average, all questions 
will be presented, otherwise you do not 
have to do all of the questions. 



k 

k 

k 

k 

k 

k 

k 

k 

k 

k 

k 

k 

k 
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press any key to continue ” 
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k k 

* THE FOLLOWING ARE QUESTIONS * 

* FROM * 

k k 

* " STUDENT PILOT FLIGHT MANUAL " * 

k k 
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" press any key to continue " 



: 1 
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" STUDENT 


PILOT 


FLIGHT 


MANUAL " 


k 


SELECT 


ONE OF 


THE FOLLOWING 


k 

k 


TYPE 1 




PART - 


ONE 


k 


TYPE 2 




PART - 


TWO 


k 


TYPE 3 


. .for 


PART - 


THREE 


k 



TYPE 0 for SELECTING -ANOTHER CONCEPT * 

k 



kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk 



kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk 
k k 

* THIS IS THE BEGINNING OF THE LESSON * 

k k 

kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk 



" press any key to continue " 



1) Which one of the following statements is correct concerning "torque" 
corrections (U.S. -built engines)? 

1 In a climb, left rudder is necessary to keep the airplane 

straight. 

2 In a climb, right rudder is necessary to keep the airplane 

straight. 

3 In a dive, right rudder is necessary to keep the airplane 

straight. 

4 When power is added abruptly, the airplane will tend to 

roll to the right. 



Your answer is 1 
* You are wrong, answer is 



2) Which one of the following statements concerning the Four Forces 
is correct? 

1 In straight and level flight, all Four Forces have 

equal values . 

2 In a climb, lift is greater than weight. 

3 In straight and level unaccelerated flight, thrust is 

greater than drag. 

4 Lift acts perpendicular to the relative wind, whether 

the airplane is climbing, flying, flying straight and 
level, or gliding. 



Your answer is 4 
^ You are correct 
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(* * removed question 3 through question 13 *) 

14) You have just completed the run-up. Just before starting to taxi 
into the take-off position, you realize that the fuel gage shows 
the tank you ran up on is very low in fuel while the other is 
nearly three quarters full. You recall this was that the case when 
you visually checked the tanks, but you forgot to put the selector 
on the fullest tanks. You should 

1 switch tanks and continue the take-off. 

2 keep the airplane on the tank you did the run-up on and 

go ahead and take-off. 

3 switch tanks and run the engine up to at lease 1500 rpm 

for a minute or more, or redo the mag and carburetor heat 
check. 

4 do none of the above. 



Your answer is 3 
You are correct 



number question in critical set = 14 

number correct answers = 13 

number incorrect answers = 1 

The probability of your ability is = 0.995 



You have not yet completed all the questions. 

You will now.be evaluated on what you have done so far. 

THERE ARE 45 QUESTIONS. 

14 critical questions. 

16 complementary questions. 

15 supplementary questions. 

you have completed only the critical questions set. 
correct answers = 13 

incorrect answers = 1 

" YOU ARE ABOVE AVERAGE, EXCELLENT! " 

* * Be careful on easy questions! ** 

You have done 1 mistakes about physical question. 



********************************************** 

* YOUR KNOWLEDGE IN THE PREVIOUS LESSON * 

* IS SATISFACTORY * 

* YOU ARE FINISHED WITH THIS LESSON * 

k k 

kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk 

" press any key to continue " 



kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk 
k k 

* THIS IS THE END OF THE LESSON * 
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DO YOU WANT TO SEE THE RESULTS 

YES PRESS " ANY KEY " 

NO PRESS ” N " 
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" ANY KEY " to continue 


k 


k 

k 




on the selected concept 
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" N " to select another lesson 
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or go to exit 
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* PLEASE SELECT ONE OF THE FOLLOWING * 

* * 

* 1 STUDENT PILOTS FLIGHT MANUAL * 

* * 

* 2 PRIVATE PILOT STUDY GUIDE * 

k k 

* 3 INSTRUMENT FLIGHT RULES * 

k k 

* 0 for " QUIT " (exit program) * 



2 (Type only number you want ) * 

****************************************************** 
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* FINAL SECTION: * 

* SCORES AND COMMENTARY * 

k 
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press any key to continue 
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SCORES AND COMMENTARY 

" STUDENT PILOT FLIGHT MANUAL " 

"PART - ONE" (Before Flight) 

THERE ARE 45 QUESTIONS. 

14 critical questions. 

16 complementary questions. 

15 supplementary questions. 

you have answered only the critical questions set. 
Total correct answers = 13 

Total incorrect answers = 1 

" YOU ARE ABOVE AVERAGE, EXCELLENT! " 

Your last test was on 10 MAY 1987. 

And the grade was 51.110. 

Your grade today was 92.86. 

Good, you are improving! 

Following is a report of your performance 

The Airplane and How It Flies 



Pts = 1, Total = 


2, 


Percent 


Grade 


= 


50.00 


Cockpit: Instruments and Systems 

Pts = 3, Total = 3, Percent 


Grade 


= 


100.00 


Preflight Check 

Pts = 1, Total - 


1, 


Percent 


Grade 




100.00 


Starting The Airplane 

Pts = 3, Total = 


3, 


Percent 


Grade 


= 


100.00 


Taxiing 

Pts = 3, Total = 


3, 


Percent 


Grade 




100.00 


Pretake-off or Cockpit Check 
Pts = 2, Total = 2, 


Percent 


Grade 


= 


100.00 


* YOUR PROBLEMS AND SUGGESTIONS FOR IMPROVEMENT 


Question about Procedures 
Pts = 5, Total = 


's. 


Percent 


Grade 


— 


100.00 


Question about Calculation: 
Pts = 1, Total = 1, 


Percent 


Grade 


— 


100.00 



Physicals questions: 

Pts = 7, Total = 8, Percent Grade = 87.50 

You should review as follows 
1) 'Torque* * see chapter 2 ,fig. 2-9, SPFM. 

** Be careful on easy question! ** 

You missed 1 of them in the total of 4. 



*********************************** 
* * 

* HOPE TO SEE YOU AGAIN * 

•k k 

kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk 



...execution ends 
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APPENDIX C 

DEMONSTRATION OF WEAK-STUDENT PERFORMANCE 



****************************************************** 
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* 0 for " QUIT " (exit program) * 
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In each test which you will work on 

I There are 3 sets of questions. 

I Each set is different in level of difficulty 

I The most difficult set will be presented 
first and then the next two sets. 

I You will be evaluated after completing each 
of the first two set of questions. 

I If you are about average, all questions 
will be presented, otherwise you do not 
have to do all of the questions. 
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* THE FOLLOWING ARE QUESTIONS * 

* * 

* FROM * 

* * 
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* " STUDENT PILOT FLIGHT MANUAL " * 

* SELECT ONE OF THE FOLLOWING * 
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TYPE 3 for PART - THREE * 

TYPE 0 for SELECTING ANOTHER CONCEPT * 
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***************************************** 
k k 

* THIS IS THE BEGINNING OF THE LESSON * 

k k 

kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk 



1) which one of the following statements is correct concerning "torque" 
corrections (U.S. -built engines)? 

1 In a climb, left rudder is necessary to keep the airplane 

straight. 

2 In a climb, right rudder is necessary to keep the airplane 

straight. 

3 In a dive, right rudder is necessary to keep the airplane 

straight. 

4 When power is added abruptly, the airplane will tend to 

roll to the right. 



Your answer is 1 



You are wrong, answer is 2 * 



(* removed question 2 through question 13 *) 

14) You have just completed the run-up. Just before starting to taxi 
into the take-off position, you realize that the fuel gage shows 
the tank you ran up on is very low in fuel while the other is 
nearly three quarters full. You recall this was that the case when 
you visually checked the tanks, but you forgot to put the selector 
on the fullest tanks. You should 

1 switch tanks and continue the take-off. 

2 keep the airplane on the tank you did the run-up on and 

go ahead and take-off. 

3 switch tanks and run the engine up to at lease 1500 rpm 

for a minute or more, or redo the mag and carburetor heat 
check. 

4 do none of the above. 
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